A Word of Caution

Dan Ariely

Big Data is like teenage sex:

everyone talks about it

nobody really knows how to do it

everyone thinks everyone else is doing it

so everyone claims they are doing it.

Terminology

Different meanings

Machine Learning

Artificial Intelligence

Statistical Learning

Applied Statistics

Historical context important

ML primarily from CS / EE

‘Engineering’ mentality

Data Format

Tabular based

##    default student   balance    income
## 1       No      No  729.5265 44361.625
## 2       No     Yes  817.1804 12106.135
## 3       No      No 1073.5492 31767.139
## 4       No      No  529.2506 35704.494
## 5       No      No  785.6559 38463.496
## 6       No     Yes  919.5885  7491.559
## 7       No      No  825.5133 24905.227
## 8       No     Yes  808.6675 17600.451
## 9       No      No 1161.0579 37468.529
## 10      No      No    0.0000 29275.268
## 11      No     Yes    0.0000 21871.073
## 12      No     Yes 1220.5838 13268.562
## 13      No      No  237.0451 28251.695
## 14      No      No  606.7423 44994.556
## 15      No      No 1112.9684 23810.174

Exchangeability

Predictive Focus

Predictive accuracy

De-emphasises inference / uncertainty / explainability

Discoverability of model parameters

Example of linear models

Production

Scaling issues

Automated ML pipelines

Software engineering

Overfitting

Bias-Variance Tradeoff

\[\begin{eqnarray*} \text{Bias} &=& \text{under-complexity error} \\ \text{Variance} &=& \text{over-complexity error} \end{eqnarray*}\]

Cross-validation

Training-test split

\(k\)-fold

Train-validation-test split

Labelled data

\[ \begin{eqnarray*} \text{Discrete output} &\rightarrow& \text{Categorisation} \\ \text{Continuous output} &\rightarrow& \text{Regression} \end{eqnarray*} \]

Linear Models

Assumes data follows distributional form

Linear in parameters

Decision / Regression Trees

##    default student   balance    income
## 1       No      No  729.5265 44361.625
## 2       No     Yes  817.1804 12106.135
## 3       No      No 1073.5492 31767.139
## 4       No      No  529.2506 35704.494
## 5       No      No  785.6559 38463.496
## 6       No     Yes  919.5885  7491.559
## 7       No      No  825.5133 24905.227
## 8       No     Yes  808.6675 17600.451
## 9       No      No 1161.0579 37468.529
## 10      No      No    0.0000 29275.268
## 11      No     Yes    0.0000 21871.073
## 12      No     Yes 1220.5838 13268.562
## 13      No      No  237.0451 28251.695
## 14      No      No  606.7423 44994.556
## 15      No      No 1112.9684 23810.174

Simple to understand

Highly explainable

Prone to overfitting

Random Forest

Ensemble of trees

Aggregate low-bias trees to reduce variance

Sample of rows, constrain splits

Self-tuning (mostly)

Boosting

Ensemble of trees

Aggregate low-variance trees to reduce bias

Probably most performant approach

Tuning more involved

Support Vector Machines (SVM)

Geometric method

Divides ‘feature space’ into regions

–

Neural Networks

Unlabelled data

Clustering

Real-world Example

Topic Modelling

Dimensionality Reduction

Entity Extraction

IRISHMEN AND IRISHWOMEN: In the name of God and of the dead generations from which she receives her old tradition of nationhood, Ireland, through us, summons her children to her flag and strikes for her freedom.

Latent Dirichlet Allocation (LDA)

Unsupervised (clustering)

Topic modelling

Lots of functionality

Assign “topics” to each “document”

word2vec

Words as vectors

Semantic meaning

\[ \text{King} - \text{Male} + \text{Female} \approx \text{Queen} \]

\[ \text{Paris} - \text{France} + \text{UK} \approx \text{London} \]

Thank You

mcooney@describedata.com

https://github.com/kaybenleroll/data_workshops